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RELATED APPLICATION 

This application is a continuation in part of U.S Serial No. 
(unaoaignc d (09/415, 956 ) , filed Oct, 8, 1999, entitled A SOCIAL INTERFACE FOR 
INTERNET/TELEVISION, which is based on a provisional patent application no. 
60/103,765, filed October 9-, 1998. 

BACKGROUND 

The field of the invention is related generally to computing devices; 
internetworking, television, television-based entertainment (including 
prerecorded, recorded on request, and live broadcast television viewing); the 
Internet and television broadcast services {including over-air broadcast, 
cable and satellite television systems) . More particularly, the field of the 
invention relates to an interactive multi media user interface (MMUI) , which 
interacts with a user and conveys information through affinity character 
based presentation of Internet and broadcast media based content and is 
capable of aggregating second level content including ecommerce opportunities. 

The niomber of homes, businesses and individuals using personal and 
portable computing devices has increased substantially. The Internet has 
become pervasive. Television is evolving into an enhanced experience where 
broadcast and scheduled programming can be supplemented with interactive 
components (typically using Internet related technologies such as HTML, 
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JavaScript, et. al.). The rooont d evelopment of standards (such as ATVEF - 
Advanced Television Enhancement Forum ATSC DASE, and Europco Europe' s DVB) to 
allow these Internet technologies to synchronize the presentation of their 
content and software activities with traditional and non-traditional 
broadcast content permits the construction of a new generation of 
applications . 

Chat has been a very popular medium at various online services (notably 
on the open Internet IRC networks such as Undernet and at services like AOL 
and Yahoo!) used as a mechanism for ad hoc communication between individuals. 

The invention of a method for chat interaction using a simple 
television remote control ? ?cliGk chat? ? — Click Chat™ (Zenith, US PATENT 
#6,519,771, issued Feb. 11, 2003) .which allows viewers in command of a 
conventional television remote control or other simple selection device to 
respond to dialog without requiring the use of a keyboard. As the dialog 
proceeds, senders can incorporate optional responses into the chat stream 
using a simple encoding. Viewers can customize their own responses to 
arbitrary questions, statements or exclamations. These viewer-defined 
responses are offered to the viewer when the click chat Click Chat™ system 
identifies a question, statement or exclamation in the chat system (in the 
simplest case by the detection of a query [?], period [.], or exclamation 
mark [!] in the incoming chat stream) . 

TV Chat is designed to support chat-based applications on the Microsoft 
Web TV® platform provides the client platform to enable the current invention 
(and the embodiments of the invention known internally as Chattercast™ and 
Future Radiaf- r" - please note that TV Chat™, Click-Chat™, Chattercast ™ and 
Future Radio™ are trademarks of PEAR AVENUE, INC. formerly known as The Kiss 
Principle, Inc.). Chattercast™ is a more conventional chat experience where 
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pre-scripted dialog is used to hide connection latency and connection state, 
to provide the viewer, in a mix of live and scripted dialog, with a seamless 
contextual dialog experience concurrent to television viewing, and to enhance 
advertising. Future Radio™ is a fuller embodiment of the invention, including 
a universe of characters that not only entertain and inform as they 
communicate with the viewer but also are able to provide an interface to 
applications and services. 

Chattercast™ scripts are parameterized so that information from a 
database can be incorporated into scripted idioms and present content 
metadata in the manner and style of the chosen character. This metadata 
typically includes information related directly to the current content, 
alternative content, subsequently available content, or a supplement to 
content. Metadata typically includes the title, authors, participants, length, 
availability, a summary, review or other similarly associated information. 

The presentation of information regarding programmed content 
(television shows and internet events), guidance through and supplemental 
enhancement of content (including advertising) , messaging, online banking, 
educational applications, e-commerce, are among the applications and services 
that can be represented by the interface. Traditionally, information 
regarding programmed content has been available through an Electronic 
Programming Guide. Interfaces to messaging applications, online banking, 
educational applications, e-commerce and so forth have been static and driven 
by functional forms and so-called dialog boxes. Future interfaces, if our 
vision is fulfilled, will be more entertaining and transactions with the 
interface will be contextual conversations. 
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Four primary coimnunication architectures are evolving in the industry 
upon which the current invention can be delivered. The first is the 
architecture represented by the first Web TV Plus® deployment s- r (please note 
that Web TV® is a registered trademark of Microsoft Corporation) . This 
architecture combines an Internet client with a slow connection via a 
telephony modem with hardware that permits the reception of Television 
broadcasts in which ATVEF triggers are present. The second architecture is an 
extension of the first and permits the client to also receive broadband 
broadcasts of data (typically via satellite) . The third architecture is 
broadband —i.e., a fast connection in both directions. The fourth 
architecture is wireless (permitting mobility) . 

These communications architectures are complicated by the storage 
architecture of the clients. All clients enjoy a limited amount of random 
access memory (ROM, RAM and FLASH) while some also carry a mass storage 
device such as a hard disk. We use different strategies (described in detail 
later) to ensure the availability of dialog depending on the combination of 
communication and client storage architecture. 

In our invention, we are concerned with two characteristics of these 
communication architectures—^ connection state and latency. Connection state 
is whether a network connection is established by the client. Latency is the 
time required to complete the delivery of a required communication. 

The goal of our methods is to provide the viewer with a continuous, 
instant-on, dialog experience. High latency and a disconnected state are 
hidden, to the degree possible, from the viewer. 

Social Interfaces are a recent innovation (Nass and Reeves, The Media 
Equation: How People Treat Computers, Television, and New Media Like Real 
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People and Places, 1996, Cambridge Univ. Pr.; ISBN: 1575860538) but several 
attempts to implement social Interfaces have failed to prove effective (e.g., 
Microoof to Microsof t ' s Bob) . A fundamental problem facing attempts to build 
social interfaces for computing devices in the past has been how to make the 
interface compelling enough that the user will return habitually to the 
interface. 

In ' the Future Radio™ service we add depth to the social interface by 
providing a unique and evolving fictional universe in which the social 
characters of the interface dwell and interact. This enables us to provide 
the viewer with a set of characters that cover personality types (dominant, 
submissive, friendly and unfriendly) and affinity groupings. 

Our categorization method (i.e., the means by which we collate and 
present data in the dialog) is affinity based. Developing a character or live 
personality in our universe that appeals to seniors or alternatively a teen 
sports fan helps us determine what information, applications and services to 
present to classes of viewer and how to make that presentation. 

Interface development remains plagued by the vagaries and talent of 
skilled human writers to provide compelling and stimulating dialog. However, 
in our invention, writers are directed toward the development of characters 
that prove most successful as a social interface. 

AI and its branch Natural Language was an attempt a decade ago to build 
parsers that were English grammar based. A few products were produced that 
provided English interfaces to a variety of computer applications. The hope 
was that communication with computers would become simplified and easy enough 
for the common fieR— computer illiterate person to more easily use the computer. 
These all failed to bring the desired results. ^ ^The reasons are not well 
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understood. Interfaces have improved in the last decade to the current point 
and click Graphical User Interface (GUI) provided by Microsoft (R) , Apple (R) 
and others. On the TV side, we see the emergence of TV browser interfaces. 
With the merging of TV with the PC and the Internet we are again at a 
crossroads in the design of the best "human" interface with this new 
technology. 

The interfaces designed so far have primarily 'been" designed from" the 
standpoint of conventions established in the origins of computer design. 
These conventions do not readily apply when the interface is presented to a 
broader population of individuals unfamiliar with the conventions or unsuited 
to the methodologies. Early interface idioms were not designed for access by 
a population with diverse educational and economic backgrounds. 

While the computer can retrieve only that which it has been given, the 
human has a broad capacity reconstructing interactions at will; gilding words— 
with emotion and play back the words with lyrical sound and oratorical fury, 
calling into service the entire body to support the words. It can cause the 
hands and arms to provide appropriate gestures, the face to take on the 
correct expression, the eyes to gleam in sync with the message being 
delivered and it can to this all on the fly, automatically. Thus, the 
presentation of the information can frequently be more important than the 
actual substance of the information. It is the presentation that focuses, 
maintains our attention. TV commercials attempt to draw immediate attention 
through the increase in sound volume, a cheap mechanism to obtain user 
attention. It is the fluctuation of sound volume and intensity, tone of 
voice, personal quirks and gesturers, characteristic pauses or speaker 
delivery techniques and antics that keep us absorbed and interested in media 
content. 
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All interfaces today are devoid of emotional content yet there is a 
real need and even an attempt by users to embed the emotions into their 
content. Users react positively when they receive even token emotional 
content back in messages, even in its current weak attempts to transmit an 
emotional message. Each of us has seen a 4: ) inside text to demark an 
attempt to jest or joke. An exclamation mark4 ! or BOLD letters to represent 
enthusiasm or danger. Information flow is not maximized by coming through in 
complete grammatically correct sentence structures. Information flow is 
maximized when it makes an impact on users. This can come in fits and starts, 
soft and loud, by head scratching and pausing, iambic rhythm, cadence or 
tempo. These are just some of the methods we use to convey emotion. It is 
this transmission of emotional content that is totally missing from current 
interfaces. 

Most of us at one time or another have toiled trying to teach a child 
something that we thought was important and were frustrated when the child 
lost interest and turned off and quit. Later, much to our amazement, the 
child was effortlessly using this knowledge as if it were second nature. We. 
may have learned that _^a friend showed me how to do it—. " Why is that and 
how did it happen we may ask. The answer is that a friend imparted that 
knowledge, albeit probably much less precisely than you may have, in. a way 
much less threatening to the child, perhaps more interesting but in all cases 
more efficient than you could have. Essentially this learning took place via 
a moro friendly, hence more efficient, —user interface. Teaching and learning 
is still an art form. Tolotoya T olstoy ' s firm belief was that art is the 
transmission of feelings. We have yet to learn how to do that in an 
interface, until now. 



09/504,327 Amended. March 1, 2003 



COMFORT LEVEL 

Interfaces have to be learned, they are not natural. Natural Language 
wa &technology proved unnatural to most people. Most people have an aversion 
to technology because they are not comfortable with the interface or the 
forced designed way in which they must interact with a technology 
Interfaces can make or break an otherwise very good and useful technology. 
The design of any good interface starts with understanding of the end* user 
and how boot thoy would like to intoract with a technology.^ This begins 
with the attempt to ease the learning of the functionality of the technology 
and how to interact with.it, thus the more comfortable a user is with the 
interface, the quicker they learn to use the technology and be proficient. 
Not just proficient, but to enjoy, obtain satisfaction from and master. For 
this reason, most tutorials start out very simple and progress to the more 
difficult in stages. 

Where there is encouragement, commiseration, laughter, entertainment, 
there is enjoyment, sustained participation, concentrated effort and 
maintenance of user attention. What is needed is a way to embed 
encouragement, commiseration, laughter_^ and in short the full gamut of 
emotional content into an interface to extend the dimension of mind as well 
as of sight and sound. 

All prior art interfaces are devoid of emotional content in 
presentation of information. Emotional content integrated with other content 
infuses users with knowledge they are striving to obtain through a 
interaction that the user can obtain it understand and internalize it. It is 
the emotional and personal element that captures and holds the users 
attention and is the mechanism used to immerse the user in. the interaction 
with the multi media user interface (MMUI) . Prior art has limited access to 
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and limited ways of presenting emotional content. In contrast, the invention 
MMUI engages the user in the type of emotional or personal interaction they 
are seeking to develop knowledge or entertain thcmoolvoo them by providing a 
virtual relationship with a opocific one or more affinity character-rS . 

What is needed is the infusion of emotional energy to convey knowledge 
from Internet and media space content presented in the form of the users own 
choosing T This mech'anism"may be in the form of a character provided from a 
list of a variety of celebrity, historical, personalities, stereotypical 
caricatures and the like, figures, that bring with them not only a domain of 
knowledge, but culture, mannerisms, attributes, method of speaking, 
personality, presentation techniques etc which a user can be comfortable with 
in interacting with the wob TV and PC technology. Thus the integration of 
the media content with the presentation from a personality or character of a 
users choice will contain the substance and emotional content that is missing 
from conventional GUIs. Emotional content can contain and convey the 
subliminal messages, which are vital to a uooro user's interaction with a 
MMUI. The addition of emotional content captures, maintains, and focuses a 
uooro user ' s interest. This emotional content comes through from the affinity 
character chosen by the user to deliver what ever it is the user is 
interested in. 

C0NSX7MER BEH21VI0R DATA COLIiECTION 

Present interfaces raise substantial privacy issues. These include 
gathering information on the users under the guise of the need to learn how 
best to present the user with opportunities relevant or of interest to the 
user quicker. This leads to implementation of extensive database record 
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keeping and constitute a severe invasion of privacy depending on the dept and 
detail of dossier kept on individuals. By choosing a character affiliated 
with a body of knowledge or a particular personality, a user defines their 
possible likes, dislikes, interests, needs, market segments, purchasing 
behavior, specific affinity groups, behavior associations and more. These 
are unobtrusive means by which a users potential needs and requirements can 
be better and more efficiently discovered and served. What are needed are 
unobtrusive methods by which we can identify uooro user preferences and 
needs . 

USER FRIENDLY MMUX 

What is needed is a new multimedia user interface (MMUI) whic h that is 
capable of integrating broadcast and internet content and mediating that 
content in a user friendly fashion. The MMUI also should provide a new and 
more effective means for scanning the enormous quantity of broadcast content 
and internet information and programs and must be able to filter that content 
to provide each viewer with selected and specialized content of interest to 
an individual viewer. 

What is also needed is a new MMUI which provides a user friendly way 
for a viewer to select content of particular interest and at the same time 
enable the program sponsors, internet service providers and other media 
advertisers to target specific audiences with advertising content tailored to 
the preferences of that audience. 

It would be desirable to provide a user friendly MMUI with a way to 
automatically segregate viewers into affinity groups with like preferences as 
this would greatly streamline content scanning and filtering of information. 
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with reference to a well-known character, like Buzz Light - yoar Year® 
from the popular motion picture Toy Story r® (trademarks of Pixar Inc.), it 
would be more entertaining, stimulating, and easier to focus attention on a 
subject such as cosmology mediated and presented through the character of 
Buzz liight- ycar . Year (R) . Content would be filtered somewhat to correspond 
with Bttge eBuzz's vocabulary and presented in his ebullient, overbearing 
comical manner. Without a doubt a Buzz Light - yoar Year® or Albert Einstein 
will maintain the attention and focus of a viewer and all the while 
transferring wanted information in an entreating way. 

A person interested in financial investment may prefer presentation 
through an investor personality such as Louis Rukeyser, Bob Brinker or the 
like, depending on the area of interest or personality preferred. Content is 
immersive because a user is conversing with a character he finds interesting, 
stimulating and or provocative. This conversation takes place with a 
character on the multi media device through a variety of dimensions, those 
being visual, audio and emotional. The mechanisms for visual and audio are 
well known, but the emotional dimension is carried via the character, the 
characters attributes and delivery style. These together provide immersion 
in the media interface. 

What is needed is an affinity group character interface which enables a 
user to identify with a character and affinity group such that a virtual 
relationship between the user and character can develop and grow over time. A 
user can access information that is most relevant to the user, at his option 
via an interface which represents the uooro user ' s affinity group . interests . 
A virtual relationship grows between the user and affinity group character. 
The user can interact with the character representative of the users selected 
affinity group, thereby facilitating the transfer of information. 
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SUMMARY OF THE INVENTION 

In addition to the pre-scripted interface referred to infra, a MMUI is 
provided, which comprises a plurality of characters/ each character being 
emblematic of a specific affinity group which that a user identifies with or 
is drawn to.-r In this method of interacting, the user selects an affinity- 
based character, which commands a domain or body of broadcast media or 
Internet content." This content can" be recalled, "selected," ''translated ^and 
presented to a user in the format of that particular characters presentation 
style. 

Another aspect of the invention provides e— commerce applications, since 
an affinity based character guide is capable of merging broadcast media with 
internet business opportunities by filtering second level or 
sponsor/ commercial content through users preferences or users choices based 
on knowledge about the users selected character based affinity group. The 
character conveys content from a plurality of secondary information sources 
selected and filtered through characters affinity group to the user via a 
virtual relationship. 

Viewers watch and listen to audio and video entertainment and 
informational programming using a multiplicity of individual enhanced 
television devices. These enhanced devices (clients) include computers with 
television tuners, games machines, portable enhanced television devices, 
enhanced recording devices, and televisions enhanced with set- top-box devices 
supporting an enhanced TV standard (e.g., ATVEF) or any similar capability 
for the display and presentation of content concurrent to an audio/video 
viewing capability. These devices are connected either permanently or 
intermittently to a network such as the Internet. Each client contains some 
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amount of data storage implemented by software, memory and/or —other Mass 
Storage Device. 

The client and network architecture includes clients infrequently 
connected to the network (such as via wireless or telephony modem) , mobile 
devices (such as those dovicoo uoing Java toohnology ouoh qd Jini or 
oimilar cellular telephones and PDAs ) , clients permanently connected to the 
network (such as via' Ethernet or cable modem) / clients able to" receive " 
broadcast data (such as via satellite or other wireless broadcast) connected 
also via a narrow band back channel, and any combination of the forgoing with 
more or less data storage. 

A set of servers provides sources of data and services 
(including a transient and permanent storage service) to the clients over 
these network architectures. In particular, these sources include a system 
for the delivery of formatted media (such as ATVEF) , a system capable of 
providing support for scheduled and unscheduled synchronization of data with 
the clients, and a system capable of delivering a real-time text, or audio, 
or audio/video chat stream. These sources can optionally include a system 
capable of providing deferred short messaging support, a database system 
containing information such as a viewers identity and characteristics of 
value for or about the viewer, a database system containing content 
programming data and other information (about television or otherwise) , a 
system capable of providing any other application (such as email messaging) 
or service (such as online banking-^) . 

A production service produces chat dialog for delivery to the clients 
that is contextual to programmed content, can provide information about 
programming, and/or is scripted to provide an interface to the other services 
available. 
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Concurrent to the entertainment and information client devices present, 
a user interacts with the text, audio, or audio/video chat stream. Dialog is 
overlaid on the primary content or is displayed alongside the primary content. 
The chat stream provides supplemental entertainment and informational 
programming and acts as an interface to available applications and services. 

Pre-scripted dialog (in the form of authored interactive transcripts) 
is delivered To the^clients ahead of play 6a<5k, delivered inunediately prior " 
to requirements or delivered on demand. 

Pre-scripted dialog is triggered by time scheduled play back, trigger 
events attached to the content being viewed by the viewer (for example, using 
ATVEF triggers), or in response to an interaction with the viewer. To provide 
the viewer with a continuous and instant-on dialog experience pre-scripted 
dialog is played to hide the connection state and high communication latency. 

Vi oworo V iewer ^ s responses to chat dialog are delivered in real-time or 
are deferred for subsequent delivery. For example, if the viQwcro v iewer ' s 
client is disconnected a response can be collected by the client and delivery 
deferred until the client next connects or participates in a scheduled 
synchronization with the service. Viewer responses are also deferred when a 
viewer is connected-^ for example, if the response is prompted by a broadcast 
response request (say to an advertisement) then, to avoid high traffic spikes 
on the network, the response is deferred. 

The pre-scripted dialog is associated with (contextual to) content, 
service or application. The pre-scripted dialog can refer to and provide 
information pertaining to programmed content (subsuming or supplementing the 
role of a conventional entertainment programming guide — EPG) . The pre- 
scripted dialog can refer the viewer to other content; provide links or 
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reminders to other available programmed content —for example, while watching 
one television program the dialog can refer the viewer to other content . The 
dialog may similarly capture a response from the viewer that expresses an 
interest in later viewing of supplemental content; permitting the dialog to 
present supporting material in subsequent interaction with the viewer (either 
during the same session of interaction or in subsequent sessions.) 

The pre-scripted dialog provides' support to programmed advertising and 
product placement —for example, offering to purchase or provide additional 
information on an advertised product or service on a users behalf (thus 
acting as the viewers agent) . The dialog also provides notifications of 
events of interest to the viewer these events include notification the new 
availability of programmed content (e.g., the start of a television program), 
incoming email, availability of an offer for a product the viewer has 
expressed an interest in (acting upon the viewers personal preferences) . 

Where content viewing is deferred (recorded -in products such as TiVo 
or Replay), the dialog is similarly deferred. Real-time chat runs concurrent 
to the pre-scripted dialog and is available subject to the timing and 
availability of a real-time connection. Real-time connections are scheduled 
by an encoding in the pre-scripted dialog stream. 

Pre-scripted dialog is fixed or variable. A richer experience is 
provided if the pre-scripted dialog stream provides multiple paths. For 
example, pre-scripted dialog should vary when viewing repeat showings of 
content. A client records the path taken through a previous viewing to avoid 
repetition. The pre-scripted and real-time dialog mix to form a single 
experience that is conversational in nature. Pre-scripted dialog hides 
connection latency and connection state. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and not limitation in 
the figures of the accompanying drawings, in which like references indicate 
similar olomont o lement and live chat servers in accordance with an aspect of 
the invention. 

FIG 8 io a flow diagram illuotrating a connoction managomont routine 
that ODtabliohoo valid connoctiono ooncurront to tranocript playback in 
acGordanoc with an aopoct of the invention. 

FIG Q io a flow diagram illuotrating tranocript playback triggorod by 
an onhancod tolovioion otandard trigger in acGordanoo with an aopoct of the 
invention. 

FIG 10 io a flow diagram illuotrating tranocript playback triggered by 
an application event trigger in acGordanco with an aopoct of tho invention. 

FIG 1 illustrates several Future Radio"* clients connected to a Future 
Radio^ server system and enhanced content broadcasts in accordance with an 
aspect of the invention. 

FIG 2 illustrates a Future Radio™ server architecture in accordance 
with an aspect of the invention. 

FIG 3 illustrates a Future Radio™ client architecture based on a 
Microsoft TV® platform in accordance with ah aspect of the invention. 
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FIG -37^4 is a flow diagram illuatrating tranocript playback triggorod by 
block diagram showing the primary components of a tranocript ovont — (item) 
typical client platform and its external connections in accordance with an 
aspect of the invention. 

FIG 12 io a flow diagram illuotrating a routine for tranocript playback 
in accordanGC with an aopoct of the invention. 

FIG 13 io a flow diagram illuotrating a routine for dioplaying 
tranooripto, waiting for offered rooponooo and oxcouting thorn and chQoking if 
the failure of a reoponoo rcquircD the oxocution of a further tranocript in 
accordance with an aopoct of the invention. 

FIG 1 4 io a block diagram that 5 illustrates the functional components 
of the Future Radio oerver client software in accordance with an aspect of the 
invention. 

FIG -37^15 is a flow diagram illustrating the fetching of the ncxt a 
routine for transcript ite m playback in accordance with an aspect of the 
invention. 

FIG -3r ^7 is a flow diagram illustrating a routine for displaying 
transcripts, waiting for offered responses and executing them and checking if 
the failure of a response requires the execution of a further transcript in 
accordance with an aspect of the invention. 

FIG 8 is a block diagram that illustrates the functional components of 
the Future Radio" server in accordance with an aspect of the invention. 
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FIG 9 illustrates a transcript with offered responses and how they can 
be displayed in accordance with an aspect of the invention. 

FIG jr?-10 illustrates a transcript with embedded directives, offered 
responses and associated executable functions in accordance with an aspect of 
the invention. 

FIG 4r»ll illustrates the functional relationship of the affinity 
character in the Future Radio™ Client and Future Radio™ server architecture 
in accordance with an aspect of the invention. 

FIG -3r ^l2 illustrates an example of an affinity character multimedia 
user interface and its components in accordance with an aspect of the 
invention. 
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DETAILED DESCRIPTION 

A method and apparatus for delivering a social interface over a 
network, supplemental to and concurrent with television (or any similarly 
broadcast content) are described in the following description, for the . 
purpose of explanation-r£ numerous specific details are set forth in order to 
provide a thorough understanding of the present invention. It will be evident, 
however, to one skilled in the art that the present invention may be 
practiced without these specific details. In other instances, well-known 
structures and devices are shown in block diagram form in order to facilitate 
description. 

In one embodiment, steps according to the present invention are 
embodied in machine-executable software instructions, and the present 
invention is carried out in a processor executing the instructions, as will 
be described in detail below. In other embodiments, hardwired circuitry may 
be used in place of, or in combination with, software instructions to 
implement the present invention. 

An aspect of the present invention relates to a system in which a 
computing device is integrated with an enhanced television receiver and 
display (client) and is connected to one or more servers over a network (such 
as the Internet) . The client includes a processing system that executes 
browser software, integrates broadcast media and enhanced television features. 
As will be described below, the client provides a platform to deliver an 
interactive dialog (chat) interface for viewers (a.k.a. users) that provides 
supplemental entertainment, information regarding content (programmed and 
otherwise) , enhancements to advertising, a community experience, and an 
interface to applications on the clients and available by access to the 
Internet and television broadcast services. 
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In one embodiment, an aspect of the present invention is implemented as 
a system known as Future Radio^;; on a platform known as Microsoft TV Pak® with 
TV chat^r- ^ and Chattercast™ (please note that Microsoft TV® and Microsoft TV 
Pa k® are registered trademarks of Microsoft Corporation) , Microsoft TV Pak® 
uses a standard television display to view enhanced television. Microsoft TV 
Pak® permits the construction of consumer electronic devices supporting 
enhanced television standards. TV Chat-ie- ^ and Chattercast"" are TV Pak 
application applications that permits the presentation of chat (TV Chat^) and 
the playback e #of parameterized pre-scripted dialog-: - (Chattercast™) . The 
Microsoft TV Pak-® client permits television broadcasts to be overlaid with 
HTML and JavaScript functionality, for triggers (references to JavaScript 
functions) embedded in the broadcast television signal to be received. TV 
Chat™ manages the network connection to a chat server and permits real-time 
chat dialog and Chattercast™ permits parameterized p re-scripted chat dialog 
to be displayed using the TV Pak client. 

The Future Radio™ client is implemented as an application written in 
HTML and JavaScript, it uses the API (Application Program Interface) provided 
by Chattercast™, TV Chat™ and the Internet to make connections to Future 

Radio™ servers, receive chat and playback pre-scripted dialog. Triggers 

embedded in the broadcast content synchronize and coordinate dialog with 
content. In accordance with the present invention, a viewer of a Microsoft TV 
Pak® client with TV Chat ™ and Chattercast™ can use Future Radio™ network 
applications and services provided by one or more remote Future Radio™ 
servers. The Future Radio™ network services are used in conjunction with 
software running on the Microsoft TV® client system to provide, through an 
evolving dialog, supplemental entertainment, information regarding programmed 
content (television shows and internet events), guidance through and 
supplemental enhancement of content (including advertising) . The evolving 
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dialog provides an interface to applications such as messaging services, 
online banking services, educational applications, and e— commerce. The 
evolving dialog incorporates notifications of various kinds. Dialog 
enhancements, applications and services are coordinated in the context of 
broadcast content that contains triggers to direct the course of the dialog. 
Dialog is also directed by interaction with the viewer. 

SYSTEM OVERVIEW 

FIG. 1 illustrates a configuration of the Future Radio^ network 
according to one embodiment. A number of clients 1 are coupled to a modem 
pool 2 via a direct-dial, bi-directional data connections 306a, 306b, which 
may be Telephone, ISDN, DSL, Cable, Wireless or any other similar type of 
connection. The modem pool 2 is coupled typically through a router, such as 
that conventionally known in the art, to a number of remote servers 4 via a 
conventional network infrastructure 3, such as the Internet. The Future 
Radio™ system also includes a Future Radio™ server 5 that specifically 
supports the Future Radio™ application running on the clients; i.e., the 
server 5 provides the Future Radio™ service to the Future Radio™ clients. 
Each of the clients 1 can connect to the server 5 through direct connections 
via telephone or similar direct connection capability or through the Internet 
via the modem pool 2. The modem pool 2 is a conventional modem pool, such as 
those found today throughout the world providing access to the Internet, and 
private networks, and may be provided by a local Internet Service Provider 
(ISP). 

The Future Radio™ server 5 provides bi-directional access to a content 
broadcaster 7 typically a cable head end, such as that conventionally known 
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in the art. The server 5 provides the content broadcaster 7 the Future Radio^ 
service for transmission to the clients 1 over the enhanced content broadcast 
6 typically a cable network, terrestrial broadcast, or satellite broadcast. 
The Future Radio™ service is integrated with broadcast content 6 using an 
enhanced television standard. In this embodiment, that standard is provided 
by the Advanced Television Enhancement Forum (ATVEF) that permits the 
embedding of triggers in the broadcast 6 by the content broadcaster. Such 
triggers are incorporated in either real-time by the content broadcaster 7 or 
are added in advance to content during postproduction of the broadcast 
content. Triggers in this embodiment take the form of calls to JavaScript 
function calls to functions defined and executed on the clients 1 . The , the 
Future Radio™ application inanning on the clients 1 display and otherwise 
present interactive transcript and live dialog to viewers. 

SERVER SYSTEM ARCHITECTURE 

The Future Radio™ server 5 generally includes one or more 
computer systems generally having the architecture illustrated in FIG 2. The 
illustrated architecture is only exemplary; a Future Radio™ server is not 
constrained to the illustrated architecture (for example, the architecture 
can be a more distributed system than illustrated here) . The illustrated 
architecture includes a central processing unit (CPU) 201, random access 
memory (RAM) 203, read-only memory (ROM) 202, a inass storage device 204, a 
modem 205, a network interface card (NIC) 206, and various other input/output 
devices 207. Mass storage device 204 include magnetic, optical or other 
equivalent storage medium. I/O devices 207 include any or all devices such as 
a display monitor, keyboard, cursor control device, etc. Modem 205 is used to 
communicate data to and from remote servers 4. 
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As noted above, the server 5 may actually comprise multiple physical 
and logical devices connected through distributed system architecture. 
Accordingly, NIC 206 is used to provide data communication with other devices 
that are a part of the Future Radio^ service. 

FIG. i48 illustrates a Future Radio™ server 5 and the application 
services provided to deliver live and pre-scripted transcripts (described in 
detail below) to the clients 1. Server 5 provides a synchronized remote 
application store 1405 805 that is periodically accessed by the clients 1 to 
update the local store 701, 702. 501, 502. Transcripts are produced (written) 
and managed from the production console 1401 801 and placed in the remote 
application store 1405 805 for delivery to the clients 1 local store 701. 501. 
Transcripts are also provided by the production console 1401 801 to the IRC 
chat server 1403 803 for live delivery to clients 1 and to the broadcast chat 
server 1404 804 for on-demand delivery to the clients 1, 

As noted previously, the Future Radio™ service is integrated with 
broadcast content 6 using the triggers specified by an enhanced television 
standard. The specification of those triggers —exactly what the trigger does, 
which content to include the triggers in, and precisely where in the content 
to include them -is specified by the Future Radio™ production team. That team 
uses production console 1401 801 (that may consist of multiple workstations 
attached to the server 5) and stored on the broadcast server 1404 804 for on- 
demand delivery to the content broadcaster 7 or the offline content 
postproduction team that may receive the trigger specification by any means 
including land mail. 

A deferred response manager -^4 ^806 gathers and processes responses 
that arrive at the remote application store 1 4 05 805 from the clients 1 during 
the previously mentioned client update of the remote application store 
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1405 . 805 . For example, a viewer may have made a response to a transcript 
associated with an advertisement during the course of their television 
viewing. If the client box 303 were not connected at that time to the Future 
Radio™ server 5 the response would have been stored in the clients deferred 
response store ^^03 -502 and subsequently have arrived at the server 5 remote 
application store 1405 805 during an update. The deferred response manager 
propagates those responses to their target destination —in this example case 
the destination could be a remote server 4 maintained by the advertiser and 
accessible to the Future Radio™ service via the Internet 3. Similarly, the 
deferred response manager -3^40 ^806 places replies to deferred responses in the 
remote application store 1405 805 for later propagation to the client. 

Chat server 1403 803 provides the live and interactive broadcast of 
transcripts, enabling the Future Radio™ production team using production 
console 1 4 01 801 and moderators console 1402 802 to interact with viewers in 
real-time. Chat server 1 4 03 803 also enables viewers to exchange real-time 
dialog when using the TV Chat client of the Microsoft TV®, TV Pak. In this 
embodiment, the real-time chat server 1403 803 is typically an Internet Relay 
Chat (IRC) server. IRC is a standard Internet protocol for the exchange of 
real-time chat messages any protocol providing similar functionality could 
be used. 

Broadcast chat server 1404 804 delivers transcripts on-demand to clients 
1 and processes requests from the Future Radio™ clients 1 using standard web 
server technology known in the art. 
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CLIENT SYSTEM ARCHITECTURE 

FIG. 3 illustrates a Future Radio™ client system 1 according to one 
embodiment. The client system 1 includes an electronics unit 303 (hereafter 
referred to as the Microsoft TV® box or the box 303), and ordinary television 
set 301, and a hand-held remote control 309. In an alternative embodiment to 
the present invention, the Microsoft TV® box 303 is built into the television 
set 301 as an integral unit. The box 303 includes hardware and software for 
providing the user with a multimedia graphical user interface that overlay or 
integrate alongside television content. This interface presents the Future 
Radio™ dialog by which the viewer can access presentation of information 
regarding programmed content (television shows and internet events), guidance 
through and supplemental enhancement of content (including advertising), 
messaging, online banking, educational applications, and e— commerce. The 
viewer can also simply view the dialog and interaction with it as 
supplemental entertainment . 

The Future Radio™ client system 1 uses the television 301 as a 
multimedia display device. The box is coupled to the television set 301 by a 
link 302. The link 302 includes an audio channel for generating sound from, 
the televisions speaker and a video channel in the form of RF (radio 
frequency) , S-video, composite video, or other format. The communication link 
306a, 306b between box 303 and the server 5 is telephone, cable modem, or 
Ethernet. The box 303 receives AC (alternating current) power through and AC 
power line 307. 

In an alternative embodiment to the present invention, the Microsoft 
TV® box 303 is built into the television set 301 as an integral unit, and 
further integrated with a wireless commxinication link 306 and battery power 
in place of the AC power line 307. In this embodiment, the client is reduced 
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size to create a portable embodiment of the invention —creating a PDA 
(Personal Digital Assistant) size Future Radio^ client. The software that 
operates Microsoft TV® (Windows CE®) is today used on a variety of hand held 
devices as well as computing systems in motor vehicles (Auto PC) . In a 
further alternative embodiment of the present invention, using the same 
technology, the Future Radio™ service is provided to clients located in motor 
vehicles . 

Remote control 309 is operated by the viewer in order to control the 
client system 1 to select optional responses presented by the dialog. The 
Future Radio™ service includes optional responses in the dialog for 
presentation by Microsoft TV Pak® TV Chat click chat ™ Click Chat™ enabling 
the viewer to interact with Future Radio™ services and participate in the on- 
going dialog (chat) . The box 303 receives commands from the remote control 
309 via an infrared (IR) commxinication link. In alternative embodiments, the 
link between the remote control 309 and the box 303 may be RF or any 
equivalent mode of transmission. Equally, the input device in alternative 
embodiments may be a keyboard, touch screen or other equivalent input device. 

The box 303 includes other application software that, when 
executed by a processor on the box 303, provides the viewer with an interface 
(using the multimedia and graphical capabilities of the box) that provides 
the viewer with access to the Future Radio™ services. The application 
software is automatically available upon the application of power to the box 
303. 

FIG. 4 shows the internal components of the Microsoft TV® box 303. 
Operation of the client system 1 is controlled by a CPU 401 that is in turn 
connected to other electronic circuitry 406. The CPU 401 executes software 
designed to implement features of the present invention. Other circuitry 406 
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implements the other functions of the Microsoft TV® box 303 and provides 
interconnection between the box components. Random Access Memory 403 provides 
transitory storage of executable programs and data. Flash memory 404 provides 
storage of certain programs and data storage. In one embodiment, a mass 
storage device 405 is included in the Microsoft TV® box. 

FIG. ^5 illustrates the principle internal structures of the 
Futurie Radio^ application that runs on the client box 303. A transcript store 
-&^j ^501 p rovides local storage of transcripts on the client box 303 and is 
implemented in this embodiment as a Windows CE storage partition in flash 
memory 404 or mass storage 405 —the particular storage implementation being 
dependent on the specification of the particular box 303 manufacturer. 
Typical box 303 architecture today where the enhanced content broadcast 6 is 
transported over a cable television network is likely to have smaller amounts 
of flash memory 404 but be permanently connected by a relatively fast 
connection with .low latency to the Internet 3 via a cable modem 407. A box 
303 architecture where the enhanced content broadcast 6 is trdo d transported 
by a satellite broadcast is likely to have a telephony ^^back channel modem 
407 that is relatively slow, has high latency to establish an initial network 
connection to the internet 3, and provides a larger mass storage 405 capable 
of storing many more transcripts than when implemented in limited flash 
memory storage 404. The Future Radio™ service balances the delivery of 
transcripts. Boxes 303 that are permanently connected to the Internet 3 via a 
low latency modem 407 connection and have low storage capacity are delivered 
transcripts ahead of requirements using the remote application storage 805 
noted above. In both cases however, locally stored transcripts are used to 
hide connection latencies; i.e. transcripts from local storage, if available, 
are played back while the Future Radio™ client attempts to initiate, 
reestablish, or wait for a connection to the Internet 3. 
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The JavaScript function calls that implement triggers embedded in the 
enhanced broadcast content 6 reference transcripts that are the most 
immediately accessible; that is, if the box 303 has a low latency connection 
306 established it loads transcripts on demand from the broadcast server 
1 4 0 4 804 , if the connection 306 is high latency, transcripts in local store 
■6 ^501 are loaded. 

Transcripts that may be triggered by ATVEF triggers 305 embedded 
in the enhanced television broadcast 6. Such a trigger may indicate the start 
of a particular television show or advertisement —transcripts triggered in 
this way can provide supplemental interactive entertainment ^as illustrated in 
FIG and/or information pertaining to the broadcast. Other events that may 

trigger transcript playback include events associated with other applications 
(such as a messaging [email], as illustrated in FIG i^lO or personal 
information manager [PIM] application) . It is important to note that FIG ^9 
and FIG ^10 static snapshots of a continuous dialog (chat) experience. 

Transcripts, illustrated in FIG and FIG -3r?-10, consist of a sequence 
of items -dialog 1702 1002 , offered responses 1601 901 , and directives 
1701 . 1001 . FIG ^9 shows a transcript that describes a conversation between 
Nico and Zelda and ends with a oliak chat Click Chat^ message 1601 . 901 . The 
string format shown in FIG ^9 is the one used in the current embodiment the 
format specifies separators between oliok ohat Click Chat™ groupings and their 
items 1601 901 (in this case, the tilde and vertical bar character), a token 
used to introduce directives 1701 1001 (the hash character) , All other items 
are treated as dialog. In general, and frequently to accommodate 
international fonts and non-European languages, any format that identifies an 
offered response 1601 901 , a directive 1701 1001 to the Future Radio™ client 
303, and dialog in the chat stream 1702 1002 (or pre-authored transcript) can 
be readily constructed by someone familiar with the art. FIG -3r&9 illustrates 



09/504,327 Amended. March 1, 2003 

how an offered response 1601, 901 is overlaid on the television screen 1602 902 
of the television 301 and presented to the viewer. In this embodiment, 
buttons 1603 903 are used to represent the groupings. 

In an alternative presentation of the same transcript, (TV 
Chat, noted earlier, is a multi mode interface) the dialog is not overlaid 
the current television show but is presented alongside a window containing 
the television broadcast. The viewer utilizes remote control 309 (or any^ 
other selection device) to select an item 1604. 904. FIG V^IO illustrates 
directives 1701 1001 included in a transcript for processing by the Future 
Radio™ client program illustrated in FIG -1^6. Offered responses are 
processed by the JavaScript of the interface FIG -37^7 . In this embodiment, 
the JavaScript obtains the dialog 1301 701 for display and the query to 
determine if the current dialog has an associated offered response 1302 
(click Ghat 702 (Click Chat ) from the TV. Chat API responsible for managing the 
TV Chat buffer 607. 507. 

FIG -1^6 illustrates how transcript items are processed. Each item 
is fetched 1201 601 and parsed for directives that isolate a transcript 1202, 
1211 602, 611 (deletes all other transcripts), makes a transcript atomic 1207, 
1212, 120 8 , 1213 607, 612, 608, 613 (causes the transcript to pause all other 
transcripts and prevent any new transcripts from proceeding until it 
completes) , to flush the chat buffer 1209 609 (delete all historical dialog) , 
schedule a connection 1210^ 1215 610, 615 (cause a connection to the network 
to be established at a later time to enable the reception of a broadcast chat 
from the broadcast chat server 1404 804 , or to enable a live connection to the 
chat server 1404 804 ) . Connection schedules are stored in a list of scheduled 
connections ^0 ^505 that cause the Future Radio™ client 303 to ensure a 
connection to the Internet is established and an associated transcript is 
executed. 
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Transcripts are run as threads, processing entities well known to 
someone familiar with the art, under the Windows CE operating system. FIG ^ 
6 illustrates the function of a transcript playback thread. Dialog is added 
to the TV Chat buffer -^O^SOV and subsequently displayed by the interface 
illustrated in FIG 4r3-;r7 . The interface checks for offered responses 1302 702 
and if a response is received, 1307 707 executes the response 1311 . 711 . The 
response 1311 711 will be stored in deferred response storage '&O2 -502 if the 
box 303 is not connected to the Internet 3. Any JavaScript function 
associated with the response e.g., as shown 1705 1005 , will also be executed. 

The Microsoft TV® platform and service in this embodiment 
typically maintains schedule -^ ^504 for client application storage, including 
601, — 602 , — & ^5Q1, 502 , 503 , to be updated with its supporting services (of 
which Future Radio^ is one) during the early hours of the morning. Updates 
can generally be performed at any time at either the viewers request or at an 
applications request. These updates synchronize the application stores 601> 
602 on tho cliont 3 03 with application otorago 501, 502 on the client 303 
with application storage on the server 5, 1405 805 causing transcripts to be 
propagated to the clients and deferred responses to be propagated to the 
Future Radio™ server 5 . Transcripts sent to the box 1 during these updates 
are played back when triggered by various triggers in broadcast content, 
transcripts, and applications. In another embodiment of the invention, 
triggers are associated with other content such as HTML pages available on 
the Internet and in local store. 

TV Database ^03 -503 contains data related to oohodulod the schedule 
of content programming , metadata associated with that content, and affinity 
associations , and is accessible to tranocripta transcript processing that can 
include data from the TV Database 4^503 or utilize the information in the 
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database in its processing 4 ^505 to identify Future Radio^ service events 
that are scheduled with content. 

Application Overview 

In one embodiment of the invention, Chattercast™ is an interactive 
contextual chat experience concurrent with television programming ~ 
Chattercast™ contextual dialogue enhances program sponsorship and paid 
advertising, is designed to enhance the viewers TV viewing experience and 
works both online and offline. Viewers, using only a remote control to 
respond, are led down a path of interactive dialogue that prepares them for 
interaction with the shows producers, advertisers and sponsors. 

Chattercast™ dialogue is a combination of pre-scripted and real- 
time chat —dialogue is delivered to Microsoft TV® clients before it is 
required. Triggered mass responses (for example, to advertising) are 
collected on a deferred basis. 

In a fuller embodiment, Future Radio™ produces and authors a mix 
of live and pre-scripted dialog for the purpose of delivering a social 
interface to community, entertainment, information, services and applications 
that run concurrent to broadcast content . 

Fictional characters developed by Future Radio™ dialog are 
supported by episodic stories presented in the available media —these stories 
may be presented using conventional television broadcast or an Internet based 
media. A fundamental problem facing attempts to build social interfaces for 
computers in the past has been how to make the interface compelling enough 
that the user will return habitually to use the agent as the interface to any 
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given computer system. Our method adds depth to the social interface by 
providing a unique and, evolving fictional universe in which the character 
dwells. Character development remains plagued by the vagaries and talent of 
skilled human writers to provide compelling and stimulating fiction. However, 
Future Radio™ directs writers toward the development of characters that prove 
most successful as a social interface. 

Viewers associate with the developed characters permitting Future 
Radio™ to develop affinity based categorization of content and services. 
Transcripts that represent a given character represent an aotoro actor ' s voice 
applying the well-known rules of theatre and elaborated on in the previously 
mentioned work of Nass and Reeves. 

Future Radio™ can be implemented on a Multimedia Personal Computer 
where it becomes a desktop supplement or replacement. 

Future Radio™ transcripts frequently utilize the Radio Talk Show 
metaphor whereby viewers are mostly spectators and when they wish to 
contribute to the ongoing dialog call in connect to the Future Radio™ server 
and contributes dialog via a moderator (editor) that previews the dialog 
before it is broadcast. Thus the interaction stream can be broken down into 
atomic interactions whereby the viewer can finish the program and the 
character can give notification of new information. An example of how the 
dialog interacts with a viewer is in the presentation of second level content. 
Dialog of the character and viewer can continue concurrently with the dialog 
and program on screen. Dialog provides for the introduction of the 
advertisement. An advertisement can be displayed on the MMUI and character 
then can make comments about the content on the screen. This mechanism 
allows the viewer to become an active participant in the advertisement at 
their own choosing. The freedom to choose removes any subconscious or 
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conscience sales pressure and enables a negotiation and a more personal 
connection to be made. The character can then continue dialog with viewer 
regarding the content and the advertising content can be linked directly to 
the characters dialog. Chat can also refer to the advertisement where the 
user selects an icon expressing preferences for affinity group. 



MMUI COMPRISING CHARACTERS EMBLEMATIC OF AFFINITY GROUPS 

In accordance with an aspect of the invention, a MMUI provides a 
plurality of affinity based categorizations, each categorization represented 
by a specific fictional character. Each character is emblematic of an 
affinity based category. The user then selects a particular character which 
he identifies with in accordance with the users personal preferences and 
personality. The character that the user selects acts as a demographic filter, 
and scans incoming broadcast content and determines the programs or channels 
to be provided to the users selecting that character. 

Referring to Fig ^11, Internet cloud 1 8 00 1100 represents 
all available content on the Internet and some broadcast TV content. This 
source of content is accessed by the Future Radio™ Server 1 8 02 1102 which also 
communicates bi-directionally with the Content Broadcaster 1 8 04 1104 using the 
available transmission media and standards. Content is received by the 
Enhanced Content Broadcaster 1 8 06 . 1106 . 

The Enhanced Content Broadcaster •3r»04 -1106 provides the 
Future Radio™ service after integrating the broadcast content using the 
enhanced television standard. The current standard used in the embodiment is 
the ATVEF standard, and enables the embedding of triggers in the broadcast by 
the Content Broadcaster in real-time or in advance of content type triggers. 
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The Future Radio™ service is then available to Future Radio™ clients 
1 8 10, 1110. 

Further referring to Fig ^11, a plurality of affinity group 
characters 1 8 14 1114 are displayed to a Future Radio™ client 1 8 10 1110 on a 
multimedia display 1 8 20 V 1120 . The user then selects a particular affinity 
group character which he identifies with based upon the users own preferences 
and personality. The selected affinity group character" lis then displayed on 
the multimedia display 1 8 20. 1120. Also shown on the display 1 8 20 1120 are the 
text 1 8 16, wob 1116, enhanced TV content 1 8 1 8 1118 and any broadcast content. 
The selected character acts as a filter and scans incoming content by 
selecting channels—, programming— r, or second level content —for the viewer 
to watch. The affinity character also provides a simple and effective —means 
for filtering and selecting programs to be recorded for later playback to a 
viewer. The content is naturally preferred by the user, because the selection 
of program content is based in accordance with the selected affinity group 
category and the predetermined preferences of the user that the affinity 
character represents. 

The affinity group character acts as a demographic filter which 
optimizes the selection of content presented to a viewer. Each affinity group 
character represents a lifestyle, personality type, presentation style and 
corresponding broadcast content with which a user can identify. In accordance 
with an aspect of the invention, the selection of a particular affinity group 
character provides pre selected broadcast content for real time or delayed 
recording and playback to a user. 
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The affinity group character facilitates the selection of internet 
content and broadcast content by exploiting the vicworo v iewer * s preferences. 
Each affinity group character is associated with a corresponding set of 
channels carrying specific broadcast content related to the lifestyle, 
personality, preferences or other characteristics represented by the affinity 
group character. The viewers preferences are known by the —viewers selection 
of a particular character. Thus, broadcast content —for a user can be pre 
selected automatically through selection of a particular affinity group 
character. When different content appears in the characters universe of 
content, the interaction history of the affinity group character —with a 
viewer can provide direction for selection of content based on prior 
indication of interest and non-interest in any particular area. 

This aspect of the invention —advantageously can create e— commerce 
opportunities as well as provide second level content tailored to a specific 
audience. This also provides the sponsor of programming content with crucial 
information concerning where to allocate advertising resources. 

EXAMPLES OF SECOND LEVEL CONTENT 

It will be appreciated that an aspect of the invention 
significantly enhances e — commerce opportunities since the user identifies 
with and naturally wants to view and assimilate the content mediated through 
the corresponding affinity group character which the user has selected. This 
also establishes a virtual relationship between a viewer and the -specific 
affinity character the viewer has selected —from the list of affinity-based 
characters. Referring to Figure jr&ll , the affinity group characteristics 
filter 1 8 11 1111 filters the universe of —broadcast content and information. 
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taking only the programming , content or channels relevant to the selected 
affinity group. From the implementation point of view, the producer of each 
character makes an editorial decision about broadcast content that fits the 
characters affinity group. This provides e— commerce advantages—, wherein the 
producer may bias or preferentially select content for mediation through the 
character on the basis of sponsorship. This content stream is then engaged 
with a regulation means 1 8 12 1112 which provides a means for switching 
information stream content from chat to commercial, called second level 
content, which can be directed by users_^ 

For example, a user could select on a displayed feature of a 
female affinity group character to ascertain what brand of coat, lipstick, 
tooth paste, hair coloring etc, the character is using or which appears on 
the screen. The response then could be displayed on the text portion 
1 8 16 1116 , vocalized by the affinity group character —or displayed on web— TV 
1 8 1 8 1118 portion with additional purchase or ordering information. 

In accordance with an aspect of the invention, any item can be 
selected for secondary information. Note that the secondary information is 
mediated or filtered through the affinity group parameters and reflects the 
preferences and content of the selected affinity group and thus the user. 

This advantageously enables program sponsors and advertisers to 
optimize advertising resources by directing advertising content to those 
viewers who are most likely to support the sponsors. Using methods that are 
well known, the sponsors also can determine the relative popularity of each 
affinity group character. In accordance with an aspect of the invention, the 
sponsors then can target the largest possible receptive viewer audience with 
advertising content by filtering that content through the most popular 
affinity group character. 
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For example, a viewer watching Private Ryan could select a 
feature such as a tank and —activate a dialogue with the corresponding 
affinity group character concerning tank battles of World War II, such as El 
Alemain, Kursk-Orel and so forth. The affinity group character dooont doesn* t 
overly distract the viewer; he simply can register interest. The affinity 
character also can present second level content at a later time later for 
playback after it has been selected by user 1 8 09. 1109. In this way, the 
affinity character acts as a filter for mediating content based on a users 
expressed interest. 

Future Radio^ characters would recognize viewer preferences and 
pass information on to content broadcasters for future display of content. 
From a service point of view, affinity group characters have popularity 
through use, which provides sponsors with valuable market and e— commerce 
information , 

In accordance with an aspect of the invention, an active feedback 
loop is provided at 1 8 22 1122 (Figure jr9-12) from the Future Radio™ Client 
i«4lrO-1110 to the Future Radio™ Server 1 8 02 . 1102 . This provides a means for 
determining a viewers personal preferences, while securing the viewers 
privacy. Tracking each affinity group characters popularity wd ewith multiple 
viewers provides information about the viewer ^s preferences without the need 
to resort to record keeping on a user's interactions with his computer or 
other intrusive or covert means for gathering personal information. The 
aspect of gathering information about an audience by tracking the affinity 
group character' popularity provides a non- invasive, non-intrusive means to 
collect information on a user's needs and wants without having to collect, 
assemble, store and mine large databases to determine consumer needs and 
preferences . 
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FIG 12 illustrates an example affinity character multimedia user interface 



and its components. The affinity group categories 1202 can be displayed by 
many modes, one being the display button 1204 which unrolls a list or 
universe of characters 1206 from which a user can select a character 
emblematic of a particular affinity group > This can also be accomplished 
aurally via a wireless pickup 1210 using a wireless technology such as Blue 
Tooth. This adds the multimedia interface to other control devices. A 
microphone 1912 1212 which will respond to voice commands can be used as well. 
A possible screen layout is shown -at 1900 1200 with the affinity character 
displayed in a smaller window 1914 1214 on the upper right of the screen 
1900 . 1200 . For example, when selecting a character associated with MTV-;—®, 
the MTV — shew ® how (please note that MTV® is a registered trademark of Viacom) 
is displayed —at 191 8 1218 on the major portion of the display screen with the 
text 1916 1216 displayed on the bottom of the screen. 

Feedback loop 1 8 22 1122 signals which affinity group characters are 
preferred. This provides a non-intrusive, non-invasive method of 
determining a characters popularity which is useful in determining not only 
users preferred —viewing content, but also producer, sponsor and advertiser 
content. The affinity character interface —itself is provided with a means 
for storing each characters interactions with the user. Thus—, it is the 
characters actions that are tracked, not the user, —thereby protecting the 
uooro user ' s privacy. 

It can be appreciated that the affinity group character provides 
sponsor information and ecommerce opportunities —with respect to a -fee— user 
without burdening the user with myriad choice overload. The facilitation 
works both ways, servicing the uaoro user ^ s needs for entertainment and 
information without information overload or loss of privacy, while enabling 
producers, sponsors and market researchers to obtain information to obtain 
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valuable information concerning each demographic group associated with the 
selection of —a specific affinity character. 

While this invention has been described in connection with 
what are considered to be the most practical and preferred embodiments, it is 
to be understood that the invention is not limited to the disclosed 
embodiments, but on the contrary is intended to cover various modifications 
and equivalent arrangements included within the spirit and scope of the 
appended claims. 

For example, interaction between the viewer and 
the affinity group character can be spoken, initiated by a mouse click, or by 
any other convenient means of initiating communication. Therefore, persons 
of ordinary skill in this field are to understand that all such equivalent 
arrangements are included within the scope of the following claims. 
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We claim: 



AUG 2 8 2003 

Technology Center 2100 



1. (currently amended) A multi media user interface for providing 
affinity based categorization of internet or television content comprising: 

a plurality of affinity based categorizations, each associated with a 
character emblematic of a specific category, and based on a personality type, 
historical personality, caricature, stereotypical character, celebrity or 
personality that a user identifies with or is drawn to; and 

a universe of content existing on the Internet or media broadcast space, 
wherein —each character is associated with and mediates a subset of the 
universe of content; 

means for selecting the affinity group character; and 

means for displaying or otherwise presenting the selected character and the 
content associated with the selected character on a multimedia dioplay m edia 
presentation devic e ouch aa a wob TV dioplay, hoadoot dioplay, — PDA or the 
like. ; 
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2. (original) A system according to claim 1 whereby said Internet and 
broadcast media content is filtered through the affinity group character such 
that selection of a particular character accesses a subset of internet and 
broadcast media content. 

3. (original) A system according to claim 1 whereby selection of the 
affinity character defines broadcast media and Internet content which is 
available for viewing when said character is selected, 

4. (original) A system according to claim 1 further comprising means 
for targeting broadcast programming to a desired audience by associating the 
broadcast programming with the affinity group character selected by that 
audience . 

5. (original) A system according to claim 1 further comprising means 
for providing advertising content to a target audience by associating the 
advertising content with the affinity group character selected by the target 
audience. 
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6. (currently amended) A multi media user interface including a 
plurality of affinity based characters comprising: 

a plurality of affinity based categorizations, each associated with a 
character emblematic of a specific category, such that each category 
represents an affinity group having like preferences; and 

each character is associated with a specific subset of a universe of content 
existing on the Internet or media broadcast spacer- 
feedback means associated with each character for storing character / user 
interaction information, such as the number of times the character is 
selected for viewing by users and the duration of time for each selections- 
means responsive to said feedback means for determining allocation of 
advertising content based on said stored character / user interaction 
information ; and 

means for displaying or otherwise presenting the affinity oharaotoro selected 
character and the content associated with the selected character on a 
multimedia dioplay m edia presentation devic e ouch oo a wob TV dioplay, hoadoot 
dioplay, PDA, — or the liko . . 
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7. (original) A system according to claim 6 wherein said interaction 
information includes: number of times each affinity group character is 
selected, time viewers spend interacting with each character, number of users 
selecting each character. 

8. (currently amended) A system as in claim 6 wherein feedback means 
includes active feedback means for recording a characters history of 
interactions with a user for implementing like interactions such that a 
virtual rolationohip botwcon an affinity Gharactor and uoor oatabliohod. 
repetition is avoided and content related to the interactions can be 
identified and presented later . 

9. (currently amended) A system as in claim 6 wherein feedback means 
provides information about the content preferences of users selecting a 
particular affinity group character such that user privacy is protected— by 
virtue of the affinity character proxy. 

10. (original) A system as in claim 6 wherein feedback means provides 
sponsors with commercial preferences of users selecting a particular affinity 
group character. 

11. (original) A system as in claim 6 wherein the feedback means 
further comprises means for enabling a user to interact with an affinity 
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character during the display of advertising content and through said 
interaction, effects alteration of the advertising content display. 

12. (currently amended) A system as in claim 6 wherein feedback means 
comprises means for tracking an affinity characters interaction with a user, 
such that —tracking interactions of the affinity character acts as a buffer 
to protect user privacy and anonymity-? — by virtue of the affinity character 
proxy. 

13. (currently amended) A system as in claim 6 wherein feedback 
means further comprises non- intrusive and non— privacy invasive means by 
virtue of the affinity character proxy for providing sponsors with 
information of commercial value by user selection. 

14. (original) A system as in claim 6 wherein feedback means 
provides sponsors with intelligence by which internet and broadcast viewing 
content can be filtered and advertising content presentation can be tailored. 

15. (original) A system as in claim 6 wherein the selection of the 
affinity character further comprises means for selecting and storing 
broadcast content for later playback to the user making the selection. 
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16. (currently amended) A multi media user interface comprising: 

a display for displaying internet and broadcast media content; 

a plurality of affinity group characters, provided on said display for viewer 
selection such that a viewer selects a character he identifies with or is 
otherwise drawn to, and the selected character then hosts the broadcast media 
content for the user; 

a means for associating one or more subsets of broadcast and media content 
with each affinity group character, such that selection of that character 
enables viewing of corresponding broadcast and media content associated with 
the selected character; 

means for enabling interaction between the viewer and the selected affinity 
group character wherein the interaction relates to and further amplifies the 
broadcast and media content hosted by that character. 

17. (original) A multi media user interface according to claim 16 wherein 
said interaction with the affinity group character comprises the display of 
advertising content relating to what is seen on the display. 
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ABSTRACT 



Computing devices capable of internetworking and receiving, storing and 
viewing programmed (scheduled) television and multimedia content provide a 
platform for access to a combination of Internet and Television content and 
services. On this platform, a mix of pre-scripted and live interactive dialog 
(chat with offered responses) are broadcast by a connected server or servers 
to provide supplemental entertainment, information regarding programmed 
content, enhancements to scheduled television advertising, a community 
experience, and an interface to Internet applications and services. The 
dialog provides a Social Interface allowing fictional and non-fictional 
characters to interact with viewers. The fictional and non-fictional 
characters are each emblematic of a specific affinity group categorization, 
which a user personally identifies with and is drawn to. Selection of an 
affinity character also provides a filter for selecting internet and 
broadcast content associated, with that character. Second level content is 
also associated with and mediated by each character. This provides e— commerce 
opportunities whereby sponsors can direct advertising content based upon 
factors such as the relative popularity of each character —or the demographic 
group associated with each character. 



